Skip to main content

Quick Setup - GDEE

The Google Earth Engine (GEE) – Data Extraction Engine is a Node.js-based tool developed by nCount to extract and localize large volumes of pixel-level information from GEE without encountering ‘user memory limit’ errors or computational issues.

The Data Extraction Engine (GDEE) allows you to perform large computations on GEE over multiple large feature collections by dividing the broader region of interest into fishnet grids of the necessary size. The GDEE is designed to run computations on GEE in the background iteratively over each grid and export the results to the local device seamlessly. The engine performs these computations across large collections of polygons or points (features in GEE) over many images. The core idea is to disaggregate these computations into simple processes across a small set of features.

The GDEE can perform any default GEE operation over images for a specified large feature collection and export the results in any format compatible with GEE’s export functionality. Users can also define an area of interest and extract spectral values, derived spectral indices, and other indices at the pixel level using the default GDEE framework.

GDEE is optimized to reduce data redundancy and relies on Node.js default garbage collection protocols. It sustains large-scale data exports through the GEE JavaScript API framework. Hence, users must have a valid GEE account and must generate an authentication key from the Google Console to access GDEE.

Installation​

To install GDEE, ensure the following are installed on your system:

  • Python
  • JavaScript
  • Node.js
  • VS Code

Additionally, you must:

  • Have a valid Google account with access to GEE.
  • Have an existing Google Cloud Project. Refer to this guide for instructions on creating a Google Cloud Project.

Steps for Installation of GDEE​

After successfully installing the above dependencies, follow these steps to set up GDEE:

  1. Clone the GDEE repository using the command:
    git clone "https://github.com/Forest-Economy-Alliance/gdee.git"
    cd gdee
  2. Install the node modules from the package.json file using the command :
    npm install
  3. Setup a virtual environment to hold the python packages by running the command, 
    python –m venv .gdee
  4. Activate the virtual environment and install the Python packages from the requirements.txt file. For windows - 
    .gdee/Scripts/Activate.ps1
    For MacOS /Unix/Linux - 
    source .gdee/bin/activate
  5. Install the necessary python package dependencies using the requirements.txt by executing: 
    pip install -r requirements.txt`
  6. Create the GEE access KEY as a json file from the Google Cloud Console and store it in the src directory inside the root directory. The file must be named “geeKey.json".
  7. Initiate the engine instance by running the command, 
    npm start
    In its default state, the engine starts with the Express app log, 
    "Server is running on port 5000"
  8. Call the API using a web browser or a third-party client interface like ThunderClient. Post successful authentication, the data will be sequentially stored as CSV files in the data folder.

Further Details and Instructions​

For detailed information on the default use case and instructions on feature disaggregation using fishnet grids refer to the documentation on nCount.